Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor/standalone tests #249

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Conversation

amirRamirfatahi
Copy link
Collaborator

@amirRamirfatahi amirRamirfatahi commented Dec 12, 2024

#172

Pre-submission Checklist

For tests to work you need a working neo4j and redis instance with the example dataset in docker/db-graph

  • Testing: Implement and pass new tests for the new features/fixes, cargo test.
  • Performance: Ensure new code has relevant performance benchmarks, cargo bench

@@ -2,7 +2,7 @@ use super::ROOT_PATH;
use crate::service::utils::make_request;
use anyhow::Result;

#[tokio::test]
#[tokio_shared_rt::test(shared)]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are the benefits for service tests of using the shared tokio runtime?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not using it can cause conflict when using the connectors.

@amirRamirfatahi amirRamirfatahi self-assigned this Dec 13, 2024
@amirRamirfatahi amirRamirfatahi marked this pull request as ready for review December 13, 2024 07:22
Copy link
Collaborator

@SHAcollision SHAcollision left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's looking good! Now that you are at it, could you also improve .github/workflows/test.yml to eliminate the steps that are not needed anymore? (building and running the service in the background, syncing the neo4j data, waiting for it to be available, etc).

Comment on lines +71 to +75
let should_reindex = redis_is_empty().await.unwrap_or(false);
if should_reindex {
info!("Starting reindexing process.");
reindex().await;
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic is not needed. We can do reindex().await


// Run the run-queries.sh script on the Docker host using docker exec
tokio::process::Command::new("docker")
.args(&["exec", "neo4j", "bash", "/db-graph/run-queries.sh"])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a few trade offs with this approach as:

  • We don't know where neo4j is (maybe not docker, maybe different containter name (as in pubky-docker)
  • Very long waits when you simply want to run a single test case often during development

Is there a way we can run only sync_graph() optionally? I am thinking maybe we can have a more granular control envvar than SYNC_DB, maybe an explicit SYNC_GRAPH variable ? We already know whether to reindex or not with the REINDEX envvar, this way we control both independently.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will see what I can do.
About your first point, that's very true. It was just a ton easier to do it this way for now. After all, it's for local tests and pipelines which won't be changing often.

setup(&config).await;

// make sure DBs are in sync with mock data
let sync_db_env = std::env::var("SYNC_DB").unwrap_or("false".to_string());
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reminder to add the new envvars to .env-sample with documentation of that they do

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll fix the docs. This doesn't necessarily need to be in .env-sample after it's mentioned in the how to run the tests parts of the README.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants